The present invention relates to a hierarchical image processing apparatus, a hierarchical image processing method, and a data storage medium, and more particularly, to a reduction of the amount of operational processing in a processing for compressively coding digital image signals having different resolutions and in a processing for multiplexing streams obtained by compressively coding image signals having different resolutions and storing or transmitting the multiplexed streams.
Further, the present invention relates to a data storage medium for storing the multiplexed stream obtained by multiplexing the streams, and a data storage medium for storing a program which realizes, by software, the compressive coding or multiplexing that can reduce the amount of operational processing.
As a digital image signal has huge amount of information, high coding efficiency is inevitably required to transmit or recording it. Recently, various image compressive coding techniques have been proposed, among which there is a technique carrying out hierarchical image coding which is being developed.
This technique carrying out hierarchical image coding is a method which subjects the digital image signal to compressive coding so that the user can obtain digital image signals such as the spatial-resolution image and temporal-resolution image having different resolutions from a kind of a bit stream. Such a hierarchical image coding method, for example, carries out compressive coding to the digital image data of High Definition Television (HDTV) and to the digital image data of Standard Definition Television (SDTV) simultaneously, relating them, and combines each coded image data having a different resolution to be transmitted. By this method, it is possible to regenerate digital image signal having the resolution, which the user requires, by decoding the corresponding coded image data.
A MPEG-based hierarchical image coding, which is an example of the foregoing conventional hierarchical image coding, is explained referring to figures, as follows.
FIG. 17 is a block diagram for explaining a conventional MPEG-based hierarchical coding apparatus.
This hierarchical coding apparatus 200 comprises a first compressive coding unit 1, a first resolution conversion circuit 31 and a second compressive coding unit 2. The first compressive coding unit 1 compressively codes an input digital image signal Sg. The first resolution conversion circuit 31 subjects the digital image signal Sg to resolution conversion in a way that its resolution is made a half both in the horizontal and vertical directions, to output a low-resolution image signal Lg. The second compressive coding unit 2 compressively codes the low-resolution image signal Lg. Before being input to the hierarchical coding apparatus 200, the digital image signal Sg is divided into frames, and each frame is divided into coding units which are two-dimensional blocks of prescribed size.
The second compressive coding unit 2 comprises a predictive processing unit 2a, a subtraction processing unit 20a, an information compression unit 2b, i.e., compressive coding means, and a variable-length coding unit 23. The predictive processing unit 2a receives the low-resolution image signal Lg and generating predicted data Pg2 corresponding to the data of portion of low-resolution image signal Lg that is a target of coding (hereinafter referred to as target processing data). The subtraction processing unit 20a outputs either the difference between the target processing data and the predicted data Pg2, or the target processing unit as it is, depending on a coding mode of the digital image signal Sg. The information compression unit 2b subjects an output Dg2 of the subtraction processing unit 20a to information compression and outputs compressed data Qg2. The variable-length coding unit 23 subjects the output Qg2 of the information compression unit 2b to variable-length coding and outputs coded image data Eg2 (hereinafter referred to as low-resolution coded data).
In this case, the information compression unit 2b comprises a DCT circuit 21 and a quantization circuit 22. The DCT circuit 21 subjects the output data Dg2 of the subtraction processing unit 20a to the Discrete Cosine Transform (DCT) which transforms data in spatial region to data in frequency region, and outputs a DCT coefficient Tg2. The quantization circuit 22 quantizes the DCT coefficient Tg2 output from the DCT circuit 21 and outputs a quantization coefficient Qg2.
Further, the second compressive coding unit 2 comprises an information expansion unit 2c, i.e., decompressive decoding means, and an addition processing unit 20b. The information expansion unit 2c subjects the quantization coefficient Qg2 output from the information compression unit 2b to information expansion and outputs expanded data ITg2. The addition processing unit 20b outputs either restored data Rg2 resulting from adding the expanded data ITg2 and the predicted data Pg2, or the expanded data ITg2 as it is as restored data Rg2, depending on a coding mode of the digital image signal Sg. The information expansion unit 2c comprises an inverse quantization circuit 24 for inversely quantizing the Qg2 output from the information compression unit 2b, and an inverse DCT circuit 25 for subjecting the IQg2 output from the inverse quantization unit 24 to the inverse DCT that transforms data in frequency region to data in spatial region, and outputting the expanded data ITg2.
Further, the predictive processing unit 2a comprises a frame buffer 26, a motion detecting circuit 28 and a motion compensation circuit 27. The frame buffer 26 stores the restored Rg2 output from the addition processing unit 20b. The motion detecting circuit 28 calculates a motion vector MV2 corresponding to the target processing data. The motion compensation circuit 27 obtains the predicted data Pg2 corresponding to the target processing data from the image data stored in the frame buffer 26, based on the motion vector MV2 from the motion detecting circuit 28. The hierarchical coding apparatus 200 further comprises a second resolution conversion circuit 32 for converting the predicted data Pg2 in a way to make its resolution equal to the resolution of the digital image signal Sg. The output CPg2 of the second resolution conversion circuit 32 (hereinafter referred to as resolution converted predicted data) is output to the first compressive coding unit 1.
The first compressive coding unit 1 has almost the same configuration as the second compressive coding unit 2.
That is, the first compressive coding unit 1 comprises a predictive processing unit 1a, a subtraction processing unit 10a, an information compression unit 1b, i.e., compressive coding means, and a variable-length coding unit 13. The predictive processing unit 1a receives the digital image signal Sg as a high-resolution image signal and generates predicted data Pg1 corresponding to the data of portion of high-resolution image signal Sg that is a target of coding (hereinafter referred to as target processing data). The subtraction processing unit 10a outputs either the difference between the target processing data and the predicted data Pg1, or the target processing unit as it is, depending on a coding mode of the digital image signal Sg. The information compression unit 1b subjects an output Dg1 of the subtraction processing unit 10a to information compression and outputs compressed data Qg1. The variable-length coding unit 13 subjects the output Qg1 of the information compression unit 1b to variable-length coding and outputs coded image data Eg1 (hereinafter referred to as high-resolution coded data).
In this case, the information compression unit 1b comprises a DCT circuit 11 and a quantization circuit 12. The DCT circuit 11 subjects the output data Dg1 of the subtraction processing unit 10a to DCT and outputs a DCT coefficient Tg1. The quantization circuit 12 quantizes the DCT coefficient Tg1 output from the DCT circuit 11 and outputs a quantization coefficient Qg1.
Further, the first compressive coding unit 1 comprises an information expansion unit 1c, i.e., decompressive decoding means, and an addition processing unit 10b. The information expansion unit 1c subjects the quantization coefficient Qg1 output from the information compression unit 1b to information expansion and outputs expanded data ITg1. The addition processing unit 1b outputs either restored data Rg1 resulting from adding the expanded data ITg1 and the predicted data Pg2, or the expanded data ITg1 as it is as restored data Rg1, depending on a coding mode of the digital image signal Sg. The information expansion unit 1c comprises an inverse quantization circuit 14 for inversely quantizing the Qg1 output from the information compression unit 1b, and an inverse DCT circuit 15 for subjecting the IQg1 output from the inverse quantization unit 14 to inverse DCT and outputting the expanded data ITg1.
Further, the predictive processing unit 1a comprises a frame buffer 16, a motion detecting circuit 18 and a motion compensation circuit 17. The frame buffer 16 stores the restored Rg1 output from the addition processing unit 10b. The motion detecting circuit 18 calculates a motion vector MV1 corresponding to the target processing data. The motion compensation circuit 17 obtains the predicted data Pg1 corresponding to the target processing data from the image data stored in the frame buffer 16 and the resolution converted predicted data CPg2 which is the output of the second image conversion circuit 32, based on the motion vector MV1 from the motion detecting circuit 18.
The hierarchical coding apparatus 200 further comprises a multiplexing circuit 4 for multiplexing the high-resolution coded data Eg1 output from the first compressive coding unit 1 and the low-resolution coded data Eg2 output from the second compressive coding unit 2 and outputting a multiplexed bit stream MEg.
The operation is described as follows.
When the hierarchical coding apparatus 200 receives the digital image signal Sg as it is separated into the frame units, as a video signal, initially, the first resolution conversion circuit 31 subjects the image signal Sg to resolution conversion in a way to make its vertical and horizontal resolutions a half of the respective original resolutions, and outputs the low-resolution image signal Lg. Thereafter, the low-resolution image signal Lg is subjected to coding in the second compressive coding unit 2.
To be specific, in the low-resolution image signal Lg, for its intra-frame coding frames (hereinafter referred to as I frames), such as a first frame, which are targets of coding, each target processing data is subjected to intra-frame coding frame coding without calculating the difference between the target processing data which is a coding unit in the I frame, and the predicted data.
That is, in the DCT circuit 21 the low-resolution image signal Lg is subjected to DCT for each target processing data corresponding to the two-dimensional block to be transformed to a DCT coefficient Tg2. In the quantization circuit 22 the DCT coefficient Tg2 is further transformed by quantization to a quantized coefficient Qg2 to be output to the variable-length coding circuit 23. In the variable-length coding circuit 23 the quantized coefficient Qg2 is subjected to variable-length coding, and the corresponding low-resolution coded data Eg2 is output to the multiplexing circuit 4.
At that time, in the information expansion unit 1c the quantized coefficient data Qg2 is transformed to the restored data ITg2, which is real-time data, to be output to the addition processing unit 20b. That is, in the inverse quantization unit 24 the quantized coefficient Qg2 is transformed by inverse quantization to the restored DCT coefficient IQg2. Further, in the inverse DCT circuit 25 the restored DCT coefficient IQg2 is transformed by inverse DCT to the restored ITg2.
In the addition processing unit 20b, since the restored data ITg2 corresponds to the intra-frame coding frame, the restored data ITg2 is output as it is to the frame buffer 26 without being subjected to addition, and is stored in the frame buffer 26.
Since image data generally has high intra-frame coding frame correlation, energy is concentrated into the DCT coefficients corresponding to lower frequency components for the data in frequency region, i.e., DCT coefficients, obtained by subjecting the image data to DCT. Therefore, in quantizing the DCT coefficient, it is possible to reduce the amount of data to be transmitted or recorded while holding a degradation in picture quality to a minimum by finely quantizing the DCT coefficient corresponding to the important lower frequency component while roughly quantizing the DCT coefficient corresponding to the visually unimportant higher frequency component.
Further, image signals of frames (a P and a B frame) except the intra-frame coding frame in the low-resolution image signal Lg, are subjected to predictive coding in which the image data required for prediction is obtained for each frame, the predicted data corresponding to the target processing data is calculated from the image data, and prediction error data which is the difference between the predicted data and the target processing data is coded.
To be specific, initially, in the motion detecting circuit 28 of the predictive processing unit 2a, the motion vector MV2, which is used in predicting the target processing data, is obtained in each two-dimensional block, for example, by the well-known exhaustive search. Thereafter, in the motion compensation circuit 27, a prescribed image data in the frame buffer 26 is calculated as predicted data for the target processing data, based on the detected motion vector MV2.
Thereafter, in the subtraction processing unit 20a the difference signal between the predicted data and the target processing data is obtained as prediction error data for the target processing data. The prediction error data is, as similar to the target processing data for the intra-frame coding frame, transformed to the compressed data Qg2 by information compression in the DCT circuit 21 and the quantization circuit 22. Further, the compressed data Qg2 is coded by the variable-length coding circuit 23 to be output as the low-resolution coded data Eg2 to the multiplexing circuit 4. At this time, the output Qg2 of the quantization circuit 22 is subjected to information expansion by the inverse quantization circuit 24 and the inverse DCT circuit 25 to be output to the addition processing unit 20b. In the addition processing unit 20b, since the restored data ITg2 that is the output of the information expansion unit 2c corresponds to the prediction error data, the restored data ITg2 is added to the predicted data corresponding to the target processing data to be output to the frame buffer 26.
In such an inter-frame predictive coding, since the prediction error data that is the difference between the target processing data and the corresponding predicted data is coded, energy, i.e., a data value, of the frequency region transformed by DCT is reduced compared with the case such as intra-frame coding where the digital image signal is directly coded, thereby making possible higher efficient coding.
Further, although the high-resolution image signal, that is, the input digital image signal Sg, is subjected to the almost similar coding for the low-resolution image signal Lg, the coding for the high-resolution image signal Sg in the first compressive coding unit 1 differs from that for the low-resolution image signal Lg in the second compressive coding unit 2 in the point where the image data obtained from the low-resolution image signal Lg is used when the predicted data for the target processing data of the high-resolution image signal Sg is generated.
That is, in the first compressive coding unit 1, the generation of the predicted data corresponding to the target processing data is carried out by the motion compensation circuit 17, but at the time, the motion compensation circuit 17 receives the high-resolution image data stored in the frame buffer 16 and the resolution converted predicted data that is the output from the second resolution conversion circuit 32, and then the motion compensation circuit 17 calculates the differences between the two data and the target processing data, and the data having the smaller difference is selected as reference data.
Further, the high-resolution image data is the image data that corresponds to the two-dimensional block in the previous frame before the current frame which is being a target of processing. The resolution converted predicted data is such that the image data corresponding to the two-dimensional block of the previous frame in the frame buffer 26 is subjected to resolution conversion by the second resolution conversion circuit 32 in a way in which the resolution doubles in the horizontal and vertical directions.
In such a hierarchical coding, for the digital image signal, i.e., the high-resolution image signal, that is not subjected to resolution conversion, the image data of the portion where the error between the high-resolution image signal and the low-resolution image signal is small is not required to be coded, whereby the coding efficiency can be increased.
As described above, the low-resolution and high-resolution image signals Eg2 and Eg1, which are obtained by coding the low-resolution image signal Lg and the high-resolution image signal Sg, are multiplexed in the multiplexing circuit 4, and the multiplexed bit stream MEg is output.
On the other hand, in a decoding apparatus which decodes such a multiplexed bit stream MEg, the low-resolution image, i.e., SDTV, can be regenerated by taking the coded image data of low-resolution from a kind of coded image data and decoding the same. Further, the high-resolution image, i.e., HDTV, can be regenerated by taking both of coded image data of low-resolution and high-resolution from a kind of the coded image data and decoding the same.
Thus, the user can switch low-resolution image and high-resolution image, according to the situation, to carry out the regeneration. (For example, ISO/IEC IS 13818-2: Information technologyxe2x80x94Generic coding of moving pictures and associated audio informationxe2x80x94Part 2: Videoxe2x80x9d, 1996.5)
However, in the hierarchical coding apparatus having the foregoing configuration, the motion compensation circuit 17 of the first compressive coding unit 1 receives both the high-resolution image data and the low-resolution image data, i.e., resolution converted predicted data, as reference data used in prediction, selects either which has the smaller error between the target processing data and it, as reference data, and by the use of the selected reference data, generates the predicted data for the high-resolution target processing data, so that the configuration of the motion compensation circuit 17 becomes much complicated.
Further, in decoding, the low-resolution image can be regenerated if applied to a decoding apparatus for a low-resolution image. The high-resolution image cannot be regenerated as long as the low-resolution coded data is not decoded, because the low-resolution image is used in prediction. Thus, in the decoding apparatus for high-resolution image that decodes high resolution coded data in the multiplexed bit stream coded by the conventional hierarchical coding apparatus, the configuration of the decoding apparatus for the low-resolution side must be incorporated inside the decoding apparatus for the high-resolution side, so that the amount of calculation in decoding is increased along with the scale of hardware.
Further, in the conventional hierarchical decoding apparatus, the decoding unit for low-resolution image signals and the decoding unit for high-resolution signals are required to be synchronized each other in their operation, so that the configuration of the whole apparatus becomes complicated.
It is an object of the present invention to provide the solution to those problems, which can reduce the amount of operation in the hierarchical coding and decoding for the digital image data having different resolutions, thereby obtaining a hierarchical image processing apparatus and hierarchical image processing method that can carry out efficiently the hierarchical coding and decoding by the use of simple circuit configuration.
It is another object of the present invention to provide a data storage medium for storing a multiplexed bit stream obtained by hierarchical coding which reduces the amount of operation, and a data storage medium for storing a program which realizes, by software, hierarchical coding and decoding which reduce the amount of operation.
Other objects and advantages of the present invention will become apparent from the detailed description desired hereinafter; it should be understood, however, that the detailed description and specific embodiment are desired by way of illustration only, since various changes and modifications within the scope of the invention will become apparent to those skilled in the art from this detailed description.
According to a first aspect of the present invention, a hierarchical image processing apparatus which receives a high-resolution first digital image signal and a low-resolution second digital image signal as input image signals, and subjects both of the digital image signals to hierarchical coding, said apparatus comprising: compressive coding means for subjecting the first and second digital image signals to compressive coding and outputting first and second coded image data; decompressive decoding means for subjecting the first and second coded image data to decompressive decoding and outputting first and second decoded image signals; and resolution conversion means for, based on the second decoded image signal, generating a third decoded image signal which corresponds to a specific image signal constituting a portion of the second decoded image signal and the resolution of which becomes equal to the resolution of the first decoded image signal; the compressive coding means subjecting the first digital image signal to compressive coding, using only the third decoded image signal as a reference image signal for a low resolution side, thereby making the reduction of compression efficiency of image data and the like, which are transmitted or stored, less, and simplifying the configuration of the hierarchical coding apparatus.
According to a second aspect of the present invention, the hierarchical image processing apparatus includes a compressive coding means that subjects the image signal corresponding to at least a portion of a frame, that constitutes the first digital image signal, to predictive coding which uses the third decoded image signal as a reference signal, thereby reducing the amount of operation to a great extent, for example, in the predictive coding of the image signal corresponding to the I frame for the high-resolution side.
According to a third aspect of the present invention, the hierarchical image processing apparatus includes a third decoded image signal as an image signal corresponding to an intra-frame coding frame, thereby reducing the amount of operation to a great extent, for example, in the predictive coding of the image signal corresponding to the I frame for the high-resolution side.
According to a fourth aspect of the present invention, a hierarchical image processing apparatus which receives a high-resolution first digital image signal and a low-resolution second digital image signal as input image signals, and subjects both of the digital image signals to hierarchical coding, said apparatus comprising: compressive coding means for subjecting the first and second digital image signals to compressive coding and outputting first and second coded image data; decompressive decoding means for subjecting the first and second coded image data to decompressive decoding and outputting first and second decoded image signals; and resolution conversion means for, based on the first decoded image signal, generating a third decoded image signal which corresponds to a specific image signal constituting a portion of the first decoded image signal and the resolution of which becomes equal to the resolution of the second decoded image signal; the compressive coding means replacing a specific portion of the second digital image signal with the third decoded image signal to generate a replaced image signal, subjecting the other portions of the second digital image signal except the specific portion to compressive coding, and outputting coded image data corresponding to the other portions of the second digital image signal, as second coded image data, thereby making the reduction of compression efficiency of image data and the like, which are transmitted or stored, less, and simplifying the configuration of the hierarchical coding apparatus.
According to a fifth aspect of the present invention, the hierarchical image processing apparatus of the fourth aspect of the present invention wherein the third decoded image signal is an image signal corresponding to an intra-frame coding frame, thereby reducing the amount of operation to a great extent in coding a low-resolution image signal.
According to a sixth aspect of the present invention, a hierarchical image processing apparatus which receives high-resolution first coded image data and low-resolution second coded image data as input image data, and subjects both of the coded image data to hierarchical decoding, said apparatus comprising: decompressive decoding means for subjecting the first and second coded image data to decompressive decoding and outputting first and second decoded image signals; and resolution conversion means for, based on the second decoded image signal, generating a third decoded image signal which corresponds to a specific image signal constituting a portion of the second decoded image signal and the resolution of which becomes equal to the resolution of the first decoded image signal; the decompressive decoding means subjecting the first coded image data to decompressive decoding, using only the third decoded image signal as a reference image signal for a low resolution side, thereby reducing the amount of operation and simplifying the configuration of the hierarchical decoding apparatus.
According to a seventh aspect of the present invention, the hierarchical image processing apparatus of the sixth aspect of the present invention wherein the decompressive decoding means subjects the coded data corresponding to at least a portion of a frame, that constitutes the first coded image data, to predictive decoding which uses the third decoded image signal as a reference signal, thereby reducing the amount of operation to a great extent, for example, in the predictive coding of the image signal corresponding to the I frame for the low-resolution side.
According to an eighth aspect of the present invention, the hierarchical image processing apparatus of the sixth aspect of the present invention wherein the third decoded image signal is an image signal corresponding to an intra-frame coding frame, thereby reducing the amount of operation to a great extent, for example, in the predictive coding of the image signal corresponding to the I frame for the low-resolution side.
According to a ninth aspect of the present invention, a hierarchical image processing apparatus which receives high-resolution first coded image data and low-resolution second coded image data as input image data, and subjects both of the coded image data to hierarchical decoding, said apparatus comprising: decompressive decoding means for subjecting the first and second coded image data to decompressive decoding and outputting first and second decoded image signals; and resolution conversion means for, based on the first decoded image signal, generating a third decoded image signal which corresponds to a specific image signal constituting a portion of the first decoded image signal and the resolution of which becomes equal to the resolution of the second decoded image signal; the decompressive decoding means replacing a portion of the second digital image signal with the third decoded image signal to generate a replaced image signal, and outputting the second decoded image signal the specific portion of which is replaced with the third decoded image signal, thereby simplifying the decoding of the low-resolution digital image signal and thus simplifying the configuration of the hierarchical decoding apparatus.
According to a tenth aspect of the present invention, the hierarchical image processing apparatus of the ninth aspect of the present invention wherein the third decoded image signal is an image signal corresponding to an intra-frame coding frame, thereby reducing the amount of operation to a great extent in decoding the low-resolution image signal. According to an eleventh aspect of the present invention, a hierarchical image processing method which divides a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding, and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution to compressive coding, into packets to be multiplexed, and outputs a multiplexed bit stream, wherein both the bit streams are multiplexed in a way in which the packets, in the multiplexed bit stream, corresponding to the first and second bit streams, are arranged in the order of frames when the first and second coded data are decoded in a prescribed order of frames, and the packets of a frame corresponding to both of the coded data abut each other, thereby easily detecting coded data corresponding to a necessary frame in decoding both of the bit streams.
According to the twelfth aspect of the present invention, a hierarchical image processing method which multiplexes a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and outputs a multiplexed bit stream, wherein in the multiplexed bit stream an identification flag, which shows the specific image data in the second bit stream that is necessary for decoding the first bit stream, is multiplexed with the first and second bit streams, thereby easily extracting necessary coded data from the second bit stream in decoding the first bit stream.
According to a thirteenth aspect of the present invention, a hierarchical image processing method which multiplexes a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and outputs a multiplexed bit stream, wherein in the multiplexed bit stream a prescribed flag is multiplexed with the first and second bit streams; and the prescribed flag is an identification flag showing an inserting position, in the second bit stream, for inserting the first coded data necessary for decoding the second bit stream into the second bit stream; or the prescribed flag is an identification flag showing an inserting position, in the second decoded image data, for inserting the first decoded image data obtained by expanding the first coded data necessary for decoding the second bit stream into the second decoded image data obtained by the expansion of the second coded data, thereby easily detecting the inserting position of the first coded data into the second bit stream or the inserting position of the first decoded image data into the second decoded image data, in decoding both the bit streams.
According to a fourteenth aspect of the present invention, a hierarchical image processing apparatus which receives a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and subjects both of the bit streams to hierarchical decoding, said apparatus comprising: coded data selection means for selecting necessary coded data from both of the bit streams; and image decoding means for regenerating image data by decoding the selected coded data; the coded data selection means, in case of decoding the first bit stream, supplying the coded data contained in the first bit stream, and a portion of the coded data, in the second bit stream, that is necessary for decoding the first bit stream, to the image decoding means, thereby reducing the amount of operation in decoding.
According to a fifteenth aspect of the present invention, a hierarchical image processing apparatus which receives a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and subjects both of the bit streams to hierarchical decoding, said apparatus comprising: coded data selection means for selecting necessary coded data from both of the bit streams; and image decoding means for regenerating image data by decoding the selected coded data; the image decoding means, at the time that the decoding of a portion of the coded data, contained in the second bit stream, necessary for decoding the first bit stream, is completed, outputting a signal of indicating the end of processing to the coded data selection means; andthe coded data selection means, in decoding the first bit stream, outputting the first and second bit streams to the image decoding means, and stopping outputting the second bit stream to the image decoding means by receiving the signal of indicating the end of processing, thereby removing the process for an unnecessary signal in decoding the first bit stream.
According to a sixteenth aspect of the present invention, a hierarchical image processing apparatus which receives a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and subjects both of the bit streams to hierarchical decoding, said apparatus comprising: coded data selection means for selecting necessary coded data from both of the bit streams; and image decoding means for regenerating image data by decoding the selected coded data; the coded data selection means rearranging the coded data in the first and second bit streams in a order in which the coded data is decoded, and outputting the rearranged coded data to the image decoding means, thereby reducing the amount of the coded data temporarily retained in the image decoding means, in decoding.
According to a seventeenth aspect of the present invention, a hierarchical image processing apparatus which receives a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second code data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and subjects both of the bit streams to hierarchical decoding, said apparatus comprising: coded data selection means for selecting necessary coded data from both of the bit streams; and image decoding means for regenerating image data by decoding the selected coded data; the coded data selection means rearranging the coded data in the first and second bit streams in a way to minimize a delay time that is a time elapsing since the coded data is input to the image decoding means until the coded data is subjected to decoding, and outputting the rearranged coded data to the image decoding means, thereby reducing the amount of the coded data temporarily retained in the image decoding means, in decoding.
According to an eighteenth aspect of the present invention, a hierarchical image processing apparatus which receives a first bit stream containing first coded data obtained by subjecting a first digital image signal having a first resolution to compressive coding and a second bit stream containing second coded data obtained by subjecting a second digital image signal having a second resolution which is different from the first resolution, and subjects both of the bit streams to hierarchical decoding, said apparatus comprising: coded data selection means for selecting necessary coded data from both of the bit streams; and image decoding means for regenerating image data by decoding the selected coded data; the coded data selection means making a portion of the coded data, in the second bit stream, necessary for decoding the first bit stream, included in the first bit stream, and outputting the portion of the coded data to the image decoding means, thereby carrying out the decoding of the first bit stream efficiently.
According to a nineteenth aspect of the present invention, a hierarchical image processing apparatus which subjects the multiplexed bit stream multiplexed by the hierarchical image processing method of the eleventh aspect of the present invention to hierarchical decoding, said apparatus comprising: coded data selection means for selecting coded data necessary for decoding, from the first and second bit stream; and image decoding means for decoding the selected coded data to regenerate image data, thereby carrying out the hierarchical decoding of the multiplexed bit stream multiplexed by the hierarchical image processing method of the eleventh aspect of the present invention.
According to a twentieth aspect of the present invention, a hierarchical image processing apparatus which subjects the multiplexed bit stream multiplexed by the hierarchical image processing method of the twelfth aspect of the present invention to hierarchical decoding, said apparatus comprising: coded data selection means for selecting coded data necessary for decoding, from the first and second bit stream; and image decoding means for decoding the selected coded data to regenerate image data, thereby carrying out the hierarchical decoding of the multiplexed bit stream multiplexed by the hierarchical image processing method of the twelfth aspect of the present invention.
According to a twenty-first aspect of the present invention, a data storage medium storing a bit stream wherein the bit stream contains the first and second coded image data subjected to hierarchical coding by the hierarchical image processing apparatus of the first aspect of the present invention, thereby simplifying the configuration of the decoding apparatus for decoding the bit stream.
According to a twenty-second aspect of the present invention, a data storage medium storing a bit stream wherein the bit stream contains the first and second coded image data subjected to hierarchical coding by the hierarchical image processing apparatus of the fourth aspect of the present invention, thereby simplifying the configuration of the decoding apparatus for decoding the bit stream.
According to a twenty-third aspect of the present invention, a data storage medium storing a program for executing hierarchical coding by a computer wherein the program makes the computer execute the hierarchical coding according to the hierarchical image processing apparatus of the first aspect of the present invention, thereby reducing the reduction of compression efficiency less and by software, realizing the hierarchical coding apparatus having a simple configuration.
According to a twenty-fourth aspect of the present invention, a data storage medium storing a program for executing hierarchical coding by a computer wherein the program makes the computer execute the hierarchical coding according to the hierarchical image processing apparatus of the fourth aspect of the present invention, thereby reducing the reduction of compression efficiency less and by software, realizing the hierarchical coding apparatus having a simple configuration.
According to a twenty-fifth aspect of the present inventino, a data storage medium storing a program for executing hierarchical decoding by a computer wherein the program makes the computer execute the hierarchical decoding according to the hierarchical image processing apparatus of the sixth aspect of the present invention, thereby reducing the amount of operation of hierarchical decoding and by software, realizing the hierarchical decoding apparatus having a simple configuration.
According to a twenty-sixth aspect of the present invention, a data storage medium storing a program for executing hierarchical decoding by a computer wherein the program makes the computer execute the hierarchical decoding according to the hierarchical image processing apparatus of the ninth aspect of the present invention, thereby reducing the amount of operation of hierarchical decoding and by software, realizing the hierarchical decoding apparatus having a simple configuration.